Low code cuts down on dev time, increases testing headaches

Low-code solutions have proven their value in the industry. Many companies are seeing increased time to production and lower development costs, but relying on these low-code solutions may mean sacrificing quality due to a lack of testing.

“My belief is that the developer side testing, the quality assurance team type testing that is traditionally done is not done with low code tools,” said Jason Beres, head of UX tool and senior vice president of Indigo.Design at Infragistics, a company that provides UI components. “It’s just like ‘hey, this tool is going to save us time, where can we cut out some of this time?’ Because the tool is template driven, it should deliver the experience that we want, so we’re going to cut out that testing and then where we end up getting into trouble is that the code generated isn’t testable code, it’s not code that your architects could pick up and start to look at like they would if it was a regular .NET app or Java app or Angular app and really understand it without having some in-depth knowledge of what that low code provider is doing to generate those screens and that content.”

One might assume that since the low-code vendor presumably has already tested its components, that might take some pressure off of testers. But Beres says that most testing — UX testing, unit testing, functional testing, etc — gets ignored by low-code vendors. “What they’re doing is saying ‘use our tool and we’ll provide all the stuff for you, you don’t have to worry about all those other things,’” Beres said.  

Eran Kinsbruner, DevOps chief evangelist at testing company Perforce Software, also cautioned against relying too heavily on the assumption that the low-code vendor has tested their components.  

“Low-code components, whether they were tested well or not, they are still merged into the code or pipeline of your application, which you are responsible for,” said Kinsbruner. “So let’s say from an end user perspective, if there is a bug in your application in production, the end user doesn’t care if you use the vendor to autogenerate some of the code or you do it yourself. Quality is quality. At the end of the day it’s your responsibility, so you cannot really rely on the vendor doing the testing for you.”

He does tend to give tool vendors the benefit of the doubt to believe that they provide a baseline of a component that will work out of the box. “But when you merge it and integrate it into your own business flows, it becomes your responsibility and you need to make sure that you have almost full control of the end to end user scenarios,” said Kinsbruner. 

Security is also a consideration. “Think about it, let’s say you are a bank and your application has its own backend services, databases and personal knowledge of your end users, how do you know these components were well-tested and they do not have any leaks or exposures,” Kinsbruner said. 

Another problem with relying on the low-code vendor to do the testing is that even if they’ve tested their components well, when an issue does arise, it’s harder for developers to resolve those issues because it’s difficult to get into the code. When tests are run and crashes or bugs arise, testers need the ability to go in and understand those issues, which requires a level of visibility. 

Kinsbruner refers to this as a “black box” of testing because developers don’t really have access to the source code of the tool that generated this code. 

“As the application developer or the product manager, you own the product regardless of where the code came from and how it was created,” said Kinsbruner. “So testing is testing and bugs are bugs. The only thing regarding the process is to understand the root cause to debug these failures and remediate them properly for future analyses. So let’s say you are looking at code that was originated by this low-code tool and you are finding bugs, sometimes you can fix them. Sometimes these are bugs that you are inheriting from some areas of code that you can not really control. How do you decide whether to release or not to release? How do you decide the priority and severity of these bugs, especially when there can be a mix of bugs, security bugs, bugs that you cannot really fix because it’s not your own code that created the root cause of these bugs?”

Max de Lavenne, CEO of Buildable, a custom software development firm, added that when troubleshooting, it’s best to avoid actually modifying the low-code solution itself. This is because when working with a low-code tool, it’s presumably been selected because it fits in with their project and end goals.

If that turns out to not be the case and the low-code solution starts clashing with those goals, then it may be best to start shopping around for a replacement. 

“To use a metaphor: if you take an apple and keep trying to morph it into a pear, maybe that’s a sign that you should’ve just bought a pear to begin with,” said de Lavenne.

Testers should help select vendors

According to Kinsbruner one of the pillars to a successful low code testing process is actually involving QA into the process of selecting low-code vendors. Testers should ideally know which APIs are being inherited, what the dependencies are, and what other technologies are used in the creation of the code. 

“The test automation architect or the QA manager needs to be in the know of exactly the architecture and the tool stack that is being used to build the code that they are supposed to bless at the end of the day prior to release,” said Kinsbruner.

Unfortunately, from what Kinsbruner has observed, most testers are kept in the dark from that process. 

“The tester and manager should know what the databases are and what APIs are being used or being called so they can build the test environments for their tools and their process. They also need to know about these kinds of tools and methods as well. I don’t think that’s the case today in most cases,” said Kinsbruner. 

According to Infragistics’ Beres, another challenge during the selection process is making sure the low-code tool fits into the dev team’s current way of working. Sometimes low-code tools are using technologies that are non-standard for the development team, which leads to complications. 

“If your dev team is an Angular team then no matter what, these are things you just can’t change in the world,” said Beres. “They’re not going to switch to something because the design team says so. They’re not going to switch to something because an executive says let’s use this tool to save time. They’re just going to use what they know. Just like they choose .NET or React, even though the outcome might be the same, it’s what the developers are used to using. So if you’re an Angular dev team and you know how to test and debug and run build scripts and run test scripts and run automation through GitHub or some other tool, that’s how you’re going to get the benefit of your accelerated time to market. If your low-code doesn’t integrate well with that or if it’s using something non-standard, not within the toolchain that you’re used to using, then that’s where they will fall down.”

Citizen developers fall behind in more complex testing

Ironically, the draw of low-code for many companies is that it allows anyone to build applications, not just developers. But when bugs arise citizen developers might not have the expertise needed to resolve those issues. 

“Low-code solutions that are super accessible for the end-user often feature code that’s highly optimized or complicated for an inexperienced coder to read,” said Max de Lavenne, CEO of Buildable, a custom software development firm. “Low-code builds will likely use display or optimization techniques that leverage HTML and CSS to their full extent, which could be more than the average programmer could read. This is especially true for low-code used in database engineering and API connections. So while you don’t need a specialized person to test low-code builds, you do want to bring your A-team.”

According to Isaac Gould, research manager at Nucleus Research, a technology analyst firm, a citizen developer should be able to handle testing of simple workflows. 

Eran Kinsbruner, DevOps chief evangelist at testing company Perforce Software, noted that there could be issues when more advanced tests are needed. He believes that scriptless testing tools may be the answer to enabling citizen developers to test more complex workflows. “You might be able to close the gap by visualizing, modeling the application and the test and let the tool create some of these test cases for you,” he said. 

Some low-code vendors might be able to set themselves apart by having good educational tools for citizen developers, Gould said. 

“I think the idea of testing is that you’re always gonna have errors in your code when you’re building an application,” said Gould. “The question is that with these low code platforms is in determining whether or not your error occurs in the areas where you did have to go into coding to finding more complex relationships where you needed a real developer to do it, or are you just putting the components together incorrectly. So I think having your citizen developers trained on those platforms is vitally important.”